A  Story  with  a  Happy  Ending:  Part  II 

Col.  Brian  Shimel,  USAF 

Dear  Reader,  this  story  began  in  the  November-December  2010  issue 
of  Defense  AT&.L  I  told  you  how,  in  a  land  so  much  like  our  own  no¬ 
body  could  tell  them  apart,  a  Small  Elite  Amphibious  Fighting  Team 
(SEAFT)  realized  they  had  a  problem,  a  problem  they  thought  could 
be  solved  by  a  new  portable  radar.  Unfortunately,  the  radar  cost  so 
much  that  they  had  to  abandon  their  idea.  But  fortunately,  another 
of  the  nation's  fighting  teams,  the  Above  Low  Objects  Fighting  Team  (ALOFT),  had 
noticed  the  new  radar  development  and  coveted  it.  The  SEAFT  was  happy  to  share 
with  the  ALOFT  the  information  and  cost  estimates  they'd  gathered.  The  ALOFT 
had  a  bigger  budget,  so  they  continued  the  development  of  the  radar,  creating 
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vital  charts  to  help  them  negotiate  the  many  offices  of  neces¬ 
sary  supervision  and  review.  But  as  the  ALOFT  gained  more  in¬ 
formation  about  the  radar,  it  became  clear  that  it  was  going  to 
cost  more  than  they'd  planned  to  spend.  We  left  our  hero— the 
ALOFT's  tool  buyer— grappling  with  the  problem.  We  rejoin 
him  as  he  attempts  to  work  out  a  solution. 


money.  If  the  program  were  a  little  cheaper,  the  tool  buyer 
reasoned,  it  could  be  reviewed  by  someone  lower  in  the  tool¬ 
buying  decision  chain  of  command;  so  getting  an  approval  to 
treat  the  new  radar  as  a  cheaper  program  had  a  big  advantage 
of  time  and  understanding  (but  usually  no  fewer  vital  charts). 


The  tool  buyer  had  been 
thinking  very  hard  how  to 
solve  his  problem.  When 
the  project  started  out, 
he  had  the  data  from  the 
SEAFT,  and  he  had  solic¬ 
ited  opinions  from  expert 
tool  buyers  from  the  pri¬ 
vate  sector.  Maybe  the 
tool  buyers  weren't  com¬ 
pletely  objective,  but  they 
were  very  experienced  and 
would  be  bidding  on  the 
chance  to  build  the  radar; 
so  it  had  been  a  good  idea 
to  ask  their  opinion.  And  it 
had  also  been  a  good  idea  to  follow  up  by  asking  his  own  staff 
to  confirm  the  private  sector's  experienced  experts'  estimate, 
even  though  the  staff  came  up  with  a  different  answer.  So  over 
a  period  of  a  couple  of  years,  he  had  come  up  with  three  an¬ 
swers:  (1)  the  SEAFT's  original  estimate,  based  on  incomplete 
data;  (2)  the  estimates  from  the  private  sector,  bid  to  fit  within 
the  available  budget;  and  (3)  an  independent,  objective  esti¬ 
mate  using  reliable  assumptions  and  the  latest  methodology 
that  resulted  in  a  much  higher  estimate  than  he  could  afford.  It 
was  a  true  dilemma  for  a  tool  buyer!  What  was  he  going  to  do? 

It  seemed  the  more  anyone  learned  about  building  the  new 
radar,  the  more  expensive  and  difficult  it  appeared  to  be. 
Clearly  it  was  going  to  be  very  hard  to  get  permission  to  build 
it.  So  the  professional  tool  buyer  came  up  with  an  idea  and 
sent  it  up  through  the  many,  many  layers  of  important  offices 
of  necessary  supervision  and  review— and  in  this  case,  I  have 
to  say  that  they  did  their  job. 

When  One  and  One  Makes  . . .  One 

The  tool  buyer's  idea  was  to  get  his  boss's  boss's  boss  to  tell 
his  boss  (a  person  who  had  the  authority  to  make  important 
decisions  about  really  expensive  tools)  that  the  radar  was  esti¬ 
mated  first  by  the  SEAFT  and  second  by  the  tool  buyer's  staff, 
and  not  to  mention  the  request  for  information  estimate  from 
the  private  sector.  He  decided  to  say  he'd  like  to  take  the  aver¬ 
age  of  those  two  estimates  as  the  official  value  of  the  radar 
development  through  the  next  kilometerstone.  (One  way  you 
could  tell  this  country  apart  from  ours,  however  alike  they  were 
in  other  ways,  was  that  it  had  wisely  converted  to  the  metric 
system  because  it  was  so  much  simpler.)  A  kilometerstone 
review  required  any  tool-buying  program  to  be  approved  by 
the  ALOFT  chief  tool  buyer  or  a  very  important  deputy  before 
it  could  go  forward  to  the  next  kilometerstone  to  spend  more 


As  you  probably  know,  an 
estimate  is  an  informed 
and  educated  guess  that 
is  improved  with  more  in¬ 
formation  and  education. 
For  the  tool  buyer  to  sug¬ 
gest  that  there  were  two 
estimates  would  be  to  for¬ 
get,  for  the  sake  of  conve¬ 
nience,  that  the  first  esti¬ 
mate  (done  by  the  SEAFT) 
had  been  done  a  very  long 
time  ago,  with  little  infor¬ 
mation  or  understanding 
about  developing  the  new 
radar;  and  the  second  gov¬ 
ernment  estimate  was  re¬ 
cent,  with  greatly  improved  information  and  analysis.  One  of 
the  many,  many  layers  of  necessary  offices  of  supervision  and 
review  actually  read  the  supporting  documentation  and  came 
to  a  logical  conclusion. 

That  office  told  the  professional  tool  buyer,  "No!  You  do  not 
have  two  estimates;  you  have  one  estimate." 

Making  a  Decision  Based  on  Evidence 

Now  that  stung  a  bit.  The  buyer  chafed  and  vented  to  his 
friends,  then  eventually  went  back  to  thinking.  And  the 
thoughts  he  thought!  What  he  ended  up  thinking  was  that 
his  trouble  was  caused  by  an  estimate  that  was  too  high.  So 
if  nobody  believed  the  estimate  with  the  best  information 
and  analysis,  perhaps  he  could  win  approval  for  the  amount 
of  money  to  develop  the  new  radar  from  the  estimates  pro¬ 
vided  by  the  private  sector.  So  he  built  a  chart.  It  was  a  good 
chart  with  lots  of  colors.  It  explained  the  possibility  that  the 
actual  amount  of  research,  development,  test,  and  evaluation 
money  it  would  take  to  develop  the  new  radar  would  be  at  or 
less  than  a  certain  figure.  The  evaluation  of  that  possibility 
was  done  by  his  expert  estimating  staff.  Ironically,  although 
our  hero  built  this  vital  chart  using  the  very  probability  profile 
his  own  government  estimators  created,  his  purpose  was  to 
discredit  the  estimate  it  was  based  on!  He  thought  that  by 
comparing  the  very  high  current  estimate  to  the  earlier  es¬ 
timates,  it  would  look  so  much  more  expensive  that  nobody 
could  possibly  think  the  new  radar  would  take  so  much  re¬ 
search,  development,  test,  and  evaluation  money  to  develop! 

The  professional  tool  buyer  unleashed  the  chart  on  his  critics! 
And  it  was  powerful.  It  was  so  powerful  that  everyone  who 
saw  it— that  is,  everyone  who  didn't  already  think  they  knew 
the  answer— immediately  recognized  that  the  original  SEAFT 


It  seemed  the  more  anyone 
learned  about  building  the  new 
radar,  the  more  expensive  and 
difficult  it  appeared  to  be.  Clearly 
it  was  going  to  be  very  hard  to  get 
permission  to  build  it. 
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estimate  didn't  have  the  best  information  and  couldn't  possibly 
be  right  anymore;  and  the  private  sector  contractor  requests 
for  information  were  clearly  based  on  an  over-eager  desire 
to  fit  into  a  predetermined  budget  limit.  Neither  of  the  first 
two  estimates  explained  the  true  amount  of  effort,  risk,  and 
uncertainty  involved  in  developing  the  new  radar. 

The  only  reasonable,  responsible  course  of  action  was  to  ac¬ 
cept  the  most  current,  objective,  and  informed  estimate  as  the 
basis  from  which  to  make  a  decision. 

Not  Just  Having  a  Process  but  Using  It,  Too 

If  you  think  back  to  how  the  SEAFT  developed  their  original 
estimate,  you  will  remember  they  weren't  at  all  sure  of  how 
much  work  needed  to  be  done.  The  ALOFT,  on  the  other  hand, 
were  able  to  spend  more  time  and  learn  much  more  about 
the  work  that  needed  to  be  accomplished.  That  explains  the 
large  difference  between  the  original  $150  million  research, 
development,  test,  and  evaluation  estimate  prepared  by  the 
SEAFT  and  the  80  percent  confidence  level  estimate  of  nearly 
$800  million  prepared  by  the  ALOFT  cost  analysts,  which  you 
can  see  illustrated  in  the  chart  below.  It  wasn't,  and  never  had 
been,  because  the  estimates  disagreed  with  each  other.  The 
difference  occurred  over  time  because  people  learned  more 
about  the  elements  that  made  up  the  new  radar.  It  was  a  natu¬ 
ral  growth. 

It's  not  cause  for  suspicion  but  for  recognition  that  as  time 
goes  on,  more  will  be  known  about  a  project  effort;  there  will 
be  less  risk  and  uncertainty;  and  the  answer  will  become  more 
clear  and  defined.  It  doesn't  mean  the  final  cost  at  comple¬ 
tion  will  be  less  or  any  different  at  all.  It  means  we  will  have 


more  knowledge  about  the  answer;  be  more  certain  about  it; 
and  we  will  make  a  better  prediction  that  will  be  more  likely 
to  hold  up  through  the  design,  development,  production,  and 
integration  processes. 

The  proposals  made  by  the  defense  contractors  from  the  pri¬ 
vate  sector  in  the  request  for  information  were  made  before 
the  ALOFT  engineers  and  analysts  did  much  of  their  work.  So 
they  were  based  on  more  evidence  than  the  SEAFT  had,  but 
not  as  much  as  the  ALOFT  had.  The  defense  contractors  also 
had  the  extra  Information  of  how  much  money  the  ALOFT 
had  available  to  spend  on  the  radar  development.  Naturally, 
nobody  was  really  willing  to  say  how  that  information  helped 
them  build  their  estimate,  as  it  didn't  help  explain  anything 
that  was  needed  to  build  a  new  radar,  except  maybe  how  many 
people  they  could  hire  at  one  time. 

In  exasperation,  the  professional  tool  buyer  gave  up  trying  to 
convince  his  boss's  boss's  boss  that  the  new  estimate  was 
unrealistic.  He  added  a  two-year  $50  million  technology  dem¬ 
onstration  phase  (a  subset  of  the  research,  development,  test, 
and  evaluation  phase)  to  his  schedule.  He  planned  to  do  cost 
comparison  studies,  trade-off  analyses,  and  technology  matu¬ 
rity  work  to  reduce  the  risk,  uncertainty,  and  overall  cost  asso¬ 
ciated  with  developing  the  new  radar.  He  briefed  this  planned 
effort— now  much  longer  than  expected— to  his  boss's  boss's 
boss.  He  even  invited  the  SEAFT  to  talk  about  what  they  knew 
about  the  new  radar  and  how  much  they  needed  it  to  continue 
defeating  their  nation's  enemies.  The  decision-making  boss 
was  completely  satisfied  and  congratulated  the  professional 
tool  buyer  on  his  achievement  of  correctly  identifying  the  best 
analysis  and  best  possible  course  of  action  to  develop  the  new 

radar  for  the  ALOFT. 
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"This  is  exactly  the  right  way  to 
do  this  work.  It  is  exactly  what 
the  many,  many  layers  of  impor¬ 
tant  offices  of  necessary  supervi¬ 
sion  and  review  were  asking  you 
to  do.  They  should  be  thrilled!" 
said  his  boss's  boss's  boss.  And 
they  were.  The  professional  tool 
buyer  accepted  the  praise  and 
looked  very  determined  to  get 
the  cost  of  the  radar  down,  if  it 
was  the  last  thing  he  ever  did! 
And  maybe  he  will .... 


Shimel  is  chief,  financial  analysis 
division,  financial  management;  and 
comptroller.  Air  Force  Space  Com¬ 
mand,  Peterson  AFB.  He  manages  an 
annual  budget  of  approximately  $12 
billion  supporting  40,000  personnel. 
The  author  welcomes  comments  and 
guestions  and  can  be  contacted  at 
bnan.shimel@peterson.af.mn. 
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